DotStar: Breaking the Scalability and Performance Barriers in Regular Expression Set Matching

Regular expressions are widely used to parse data and to detect recurrent patterns and information: they are a common choice for defining configurable rules for a variety of systems. In fact, many dataintensive applications rely on regular expression parsing as the first line of defense to perform on-line data filtering. Unfortunately, few solutions can keep up with the increasing data rates and the complexity posed by several hundreds of regular expressions. In this paper we present DotStar (.*), a complete software tool-chain that can compile a set of regular expressions into an automaton that is highly optimized to run on multi-core processors with vector/SIMD extensions. DotStar relies on several algorithmic breakthroughs, to transform the user-provided regular expressions into a sequence of more manageable intermediate representations. The resulting automaton is both space and time efficient, and can perform the search in a single pass, without backtracking. The experimental evaluation, performed on a state-of-the-art Intel quad-core processor, shows that DotStar can efficiently handle both small sets of regular expressions, such as those used in protocol parsing, and much larger sets like the ones designed for Network Intrusion Detection Systems (NIDS). The experimental results show that we can achieve processing rates ranging from 2.2 Gbits/sec with the more demanding sets of NIDS expressions, to 5 Gbits/sec with XML parsing, with a performance speedup of almost two orders of magnitude when compared to popular libraries such as Boost, reaching, and in some case exceeding, the performance of specialized ASICs and FPGAs.